Linguistics 92 MULTIMODAL COMMUNICATION 2005 PROCEEDINGS from the Second Nordic Conference on Multimodal Communication

Speech and language science and technology evolved under the assumption that speech was a solely auditory event. However, a burgeoning record of research findings reveals that our perception and understanding are influenced by a speaker's face and accompanying gestures, as well as the actual sound of the speech. Perceivers expertly use these multiple sources of information to identify and interpret the language input. Given the value of face-to-face interaction, our persistent goal has been to develop, evaluate, and apply animated agents to produce realistic and accurate speech (Massaro, 1998). Baldi is an accurate three-dimensional animated talking head appropriately aligned with either synthesized or natural speech. Baldi has a realistic tongue and palate, which can be displayed by making his skin transparent. To implement multilingual agents, we have developed a client/server architecture system (Massaro et al., 2005; Ouni et al., 2005). The client is the application controlling Baldi. It sends text from a variety of languages including Arabic, Mandarin, and many European languages as well as English to a general speech synthesis server. The server generates the appropriate phonemes in the appropriate language with all the information needed by the client (phonemes, duration, pitches, word boundaries, etc.) and the acoustic speech waveform, and then it sends them back to the client. Using this information, the client generates the appropriate 1 Baldi is a registered trademark of Dominic W. Massaro.